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Abstract: Assessment of clinical competencies by direct observation is problematic for two main 
reasons: the identity of the examinee influences the assessment scores, and direct observation 
demands experts at the exact location and the exact time. Recording the performance can over- 
come these problems; however, managing video recordings and assessment sheets is troublesome 
and may lead to missing or incorrect data. Currently, no existing software solution can provide a 
local solution for the management of videos and assessments but this is necessary as assessment 
scores are confidential information, and access to this information should be restricted to select 
personnel. A local software solution may also ease the need for customization to local needs 
and integration into existing user databases or project management software. We developed an 
integrable web-based solution for easy assessment of video-recorded performances (ISEA). 
Keywords: education assessment, assessment software, video-based assessment 

Background 

Clinical performance can be assessed by expert observation using assessment sheets, 
but this is problematic for two main reasons. Firstly, the identity of the examinee can 
influence the assessment. '-^ This can be due to personal relations and prejudices,' which 
can be a large problem in smaller countries or within smaller medical specialties. 
Age and job title have also been shown to influence assessment scores.^ Secondly, 
assessments are time-consuming and need experts to be at a specific location and at a 
specific time. A feasible way of addressing these problems is by video recording the 
clinical performance. 

Recording videos can be unproblematic if designed carefully.^ However, the 
assessment procedure can easily become problematic and lead to logistic problems. 
Typically, hard drives or flash drives with videos and assessment sheets are sent to all 
examiners. When these are returned back to the data manager, some sheets may be 
incomplete and in some cases, may even be forgotten, requiring that the data man- 
ager go through a tiresome process to obtain all assessment scores. Finally, when all 
the cases are fully assessed, paper sheets are transcribed into a database in duplicate 
to avoid any typing error. This nonautomated assessment procedure is costly, time- 
consuming, and error-prone. 

Existing software enable an automated process''^ but are not optimized for video- 
recorded assessments and are problematic since data are stored at external institutions; 
instead, data should be stored locally as assessment scores are confidential information, 
and access to such should be restricted to course personnel (or study personnel, in the 
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case of research).'' ' We developed an integrable web-based 
solution for easy assessment of video-recorded performances 
(ISEA). 

Design 

The design of ISEA is based on two user interfaces; one for 
the administrator and one for the examiner. The administra- 
tor manages projects, examiners, videos, and assessment 
sheets. A "project" is defined as a group of examiners who 
assess videos based on the same assessment sheets. The 
assessment sheet is composed of questions, each having a 
number of scoring items. Within a project, examiners are 
assigned one or more videos that, if necessary, may differ 
between the examiners. Results from all assignments are 
available from the administrator interface. User profiles and 
results are stored in a local database. Depending on the legal 
requirements in each project, videos can either be uploaded 
to an external video storage platform, such as Vimeo,^ or 
stored internally within an institution, and ISEA is then 
provided with a link to the uploaded video. Examiners have 
their own personal page with a list of assigned videos. An 
assigned video is seen side-by-side with the assessment 
sheet, to enable simultaneous assessment (Figure 1). When 
the assignment is completed, it is locked and scores cannot 
be changed. The results are available for the administrator 
and can be exported in a semicolon-separated or a comma- 
separated file format (.csv). 

We used PHP: Hypertext Preprocessor (The PHP Group) 
and My Structured Query Language (MySQL) as the server- 
side scripting language and database, respectively, as these 
are the most popular and commonly used' '" and may facilitate 
an easier implementation and integration to existing software. 
We wanted the examiners to be able to use our web solution in 
any digital device anywhere, so the interfaces were developed 
using very simple Hyper Text Markup Language (HTML) 



and Cascading Style Sheets (CSS) that are compatible with 
a large range of internet browsers, including smart phones 
and tablets. No add-on software is needed for ISEA. This 
clean and simple interface may ease integration to existing 
platforms. Adobe Dreamweaver CSS (Adobe Systems, Inc., 
San Jose, CA, USA) was used for development. A download 
link for ISEA is available in the references." 

Discussion 

Video-based assessment of clinical performance may help 
eliminate bias related to the identity of the examinee by 
masking the identity, which can be achieved by adjusting 
the camera-angle or by editing the video afterwards.' '^ " 
However, the identity is rarely perfectly hidden - the face 
may be blurred and the voice may be distorted, but the iden- 
tity may still be guessed if the examiner is familiar with the 
examinee and is aware of his or her specific characteristics, 
such as manner of speech, preferences of clothes, or char- 
acteristic body figure or posture. Also, in some cases, facial 
expression, eye contact, or the finer aspects of the voice, such 
as the tone, may be relevant for the assessment itself. A step 
toward better blinding and the reduction of biases emerging 
from personal relations may therefore be achieved by, not 
only use of videos but also, collaboration with examiners 
in other institutions and other countries. Video-recorded 
assessments provide flexibility for the examiners as they no 
longer need to be at a specific location at a specific time but 
may be at any location and assess the clinical performance 
at any time.'- Implementing video-based assessments is, 
theoretically, not a problem, but practically, this is a tiresome 
process with potential pitfalls. Few studies have compared 
direct observation with video-based assessments.'^ ''' Ma et al 
assessed central venous catheterization using both direct 
observation and video recordings and found no differences 
in ratings, and they reported that importance should be given 
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to practical aspects." Kneebone et al explored the feasibility 
of a video-based assessment procedure with remotely placed 
examiners, and they also reported that a future improve- 
ment would be an automated approach for easy assessment 
of video-recorded performances.'" We developed ISEA, an 
integrable web-based solution for easy assessment of video- 
recorded performances that provides a format that can be 
stored locally, is customizable into existing systems, does not 
require any add-on software, and that is compatible with a 
large range of web browsers. ISEA may ease the assessment 
process of video-recorded performances and collaboration 
with examiners from other institutions. ISEA was used in 
the assessment process of a recent project,'^ and the general 
comment from the examiners was that ISEA was easy to use. 
Also, ISEA can be customized to local systems, for integra- 
tion to an existing project or user management systems that 
are used to administer courses and research. A few limitations 
should be noted. ISEA is an integrable application and not 
an "out of the box", stand-alone application, and the imple- 
mentation and customization may thus require some technical 
assistance. Customization, however, can be advantageous 
and even necessary for some types of education and research 
projects.' We did not describe the implementation process in 
this paper as the implementation process is more technical 
and may vary greatly between institutions, depending on the 
software used. In its simplest form and without much effort, 
ISEA can be implemented using only a user database for the 
administrator and the examiners. In our experience, imple- 
mentation has been unproblematic, and the limited technical 
assistance needed to implement ISEA is a small price to pay 
for a smooth and easy assessment process. '"^ 
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